Integration Guide Universal Robots PolyScope X BPS 1.12

This guide describes how to integrate Universal Robots controllers with PolyScope X with Bin Picking Studio (BPS) 1.12. The Robot module is a URScript library, photoneo_common_X.script, that is embedded in a set of .urpx example programs. You build your program from script nodes.

The same Robot module is also available as a URCap, which adds a settings page and BPS program nodes to the teach pendant. See Integration Guide Universal Robots PolyScope X BPS 1.12 (URCap).

Note

For the general concept of the communication, see the Robot communication overview. The supported versions of all Robot modules are listed in the Robot module support matrix.

1. Prerequisites

The Robot module for PolyScope X is delivered as the URScript library photoneo_common_X.script and a set of .urpx example programs. The library and the customer definitions are embedded in every example program. There is no URCap to install and no script file to load separately. When you import an example program, the installation is complete.

Before you start, make sure that your system meets these requirements:

• A Universal Robots e-Series robot with PolyScope X system software

• PolyScope X 10.12.1 or higher. The Robot module was developed and tested on 10.12.1. The screenshots in this guide were taken on 10.14.0.

• A free Ethernet port on the robot controller for the connection to the vision controller

• The Real-Time Data Exchange (RTDE) interface enabled on the robot controller (see section 2.2)

• A USB flash drive to transfer the .urpx example programs to the robot controller

• A vision controller with Bin Picking Studio 1.12

• A Bin Picking Studio solution with at least one calibrated vision system, a configured environment and a configured grasping method

• A MotionCam-3D for the dynamic MultiView (Instant Meshing) example. All other examples work with any supported Photoneo 3D sensor.

Note

This guide does not cover PolyScope 5 and CB-Series controllers. For them, use the separate Robot module for PolyScope 5. It has the same requests, but it uses a different program format (.urp) and keeps the scripts as files on the robot controller. See Integration guide for robots by Universal Robots.

Note

When the socket to the Action Request Server opens, the library sends the 24-byte brand ID UNIVERSAL_ROBOTS/v1.12.0. The vision controller selects the protocol features from this string, so Bin Picking Studio 1.12 or higher is required.

To check the system version on your robot, go to Main Menu → About:

The PolyScope X version is shown on the first line (Software):

2. Robot Controller Setup

2.1 Network Configuration

First, set the IP address of the robot controller port that you use for the connection to the vision controller. Go to Main Menu → Settings:

Scroll down to Connection → Network. The network settings are password protected. Press Unlock and enter the administrator password. Select Static as the network method and enter the IP address and the subnet mask of the robot controller. This guide uses IP address 192.168.1.2 and subnet mask 255.255.255.0.

Click Apply Configuration, then click Lock and Close to confirm the changes.

Note

The subnet mask 255.255.255.0 is the same as a 24-bit subnet mask (/24). For other combinations, see this table: https://dnsmadeeasy.com/support/subnet

Example of a matching network configuration on the vision controller side:

• Vision controller IPv4 address: 192.168.1.1 / 24

• Robot controller IPv4 address: 192.168.1.2 / 24 (as configured in the previous step)

You can also use the Test Connection button on the Network page of the vision controller to ping the robot controller.

Based on the result of the test (ping command), a Robot Available or Robot Unavailable message is shown. If the robot is unavailable, check the cables and the network configuration.

2.2 Enabling Real-Time Data Exchange (RTDE)

The vision controller reads the robot state (joint positions and TCP pose) directly from the robot controller through the Real-Time Data Exchange (RTDE) interface. It needs this data for calibration, hand-eye scanning and robot visualization. No Robot State Server program runs on the robot.

Go to Main Menu → Settings → Security → Services and make sure that Real-Time Data Exchange (RTDE) is enabled. The service uses port 30004.

Note

The Services page is also password protected. Press Unlock and enter the administrator password before you change an interface.

2.3 Loading the Robot Module Files

The Robot module consists of two script blocks:

• photoneo_common_X.script - the Photoneo request API (section 3.2). Do not edit it.

• the customer definitions - the bin picking procedures, the speed settings and the gripper dispatch (sections 3.3 and 3.4). Adapt this part to your application.

It also contains these example programs:

• pho_main_basic.urpx

• pho_main_basic_hand_eye.urpx

• pho_main_basic_multi_vs.urpx

• pho_main_change_enviroment.urpx

• pho_main_change_solution.urpx

• pho_main_change_bb.urpx

• pho_main_get_object_pose.urpx

• pho_main_get_status.urpx

• pho_main_reuse_scan.urpx

• pho_main_multiview_static.urpx

• pho_main_multiview_dynamic.urpx

• pho_calibration_basic.urpx

• pho_calibration_auto.urpx

• pho_calib_basic_hand_eye.urpx

Note

Copy only the .urpx programs to the robot controller. PolyScope X keeps scripts inside the program, not in separate files. Both script blocks are already embedded in every example program, as two Script nodes in the Before Start sequence. The stand-alone photoneo_common_X.script file is delivered for reference and for the update procedure in section 5.

Copy the .urpx files to a USB flash drive and plug it into the USB port of the teach pendant or of the robot controller. Then go to Main Menu → System Manager:

In the System Manager, you open, save and manage all programs on the robot. Press + Program, then Import Program, and select the example programs on the USB drive:

After the import, the example programs are listed on the Programs page of the System Manager. You can open them from there:

Note

PolyScope X does not keep robot programs as files in a /programs directory like PolyScope 5. Programs are stored in an internal application store. You cannot transfer them with scp or an FTP client. Import them from external media in the System Manager, as described above.

SSH access to the robot controller is available for diagnostics. It is disabled by default. To enable it, go to Main Menu → Settings → Security → Secure shell. There you set the port, the authentication method (password, key or both) and the authorized keys.

2.4 Tool (TCP) Setup

Warning

pho_request_scan and pho_request_trigger_scan temporarily override the active TCP. The library sets the TCP to the tool flange (p[0,0,0,0,0,0]), sends the request and then restores the previous TCP. For pho_request_scan, it restores the TCP right after it sends the request. For pho_request_trigger_scan, it restores the TCP after the vision controller confirms the request. This makes sure that the vision controller reads the flange pose over RTDE.

During this short time, get_actual_tcp_pose() called from a parallel thread returns the flange pose and not the pose of your tool. Take this into account in multi-threaded programs.

Bin Picking Studio works with a zero TCP. Set all TCP values to zero, because Bin Picking Studio compensates for the tool offsets on the vision side. Use the same setting for calibration. In the left panel, go to Application → End Effectors:

Make sure that the Tool_flange end effector has zero position and orientation values. This TCP must be active during all Bin Picking Studio operations. You can configure other end effectors for other operations, for example for placing.

Note

Check which TCP is active when you program your own motions. A taught waypoint is a pose of the TCP that was active when you taught it. Activate your own end effector for your own movej / movel motions, and switch back to Tool_flange before the next Photoneo request. If you pass a pose to pho_request_trigger_scan, use get_actual_tool_flange_pose(), which does not depend on the active TCP, and not get_actual_tcp_pose().

2.5 Robot State and Robot Pose Visualization

The UR Robot module has no Robot State Server program. When the RTDE service on the robot is enabled, the vision controller connects to RTDE port 30004 and reads the current joint positions and the Cartesian TCP pose. Nothing has to be started on the robot. The joint positions are used for robot visualization. The TCP pose is necessary for calibration and for all hand-eye scan requests.

The robot state is also useful for first configuration checks. For example, you can show the current robot and gripper state on the Environment page with the Robot module motion mode:

To check that the correct zero tool pose is reported, start the calibration, switch from the Texture tab to the Verification tab and enable Tool Pose (robot controller) in the Axis menu. The frame marker must be in the center of the flange, with the Z axis pointing down. It must be identical to the Tool0 (robot model) frame.

3. Robot Module

Note

We strongly recommend that you read the Photoneo robotic API before this section.

3.1 Connection to the Vision Controller

The program opens the connection to the Action Request Server on the vision controller at the start with pho_wait_for_server. You can send requests only after the connection is established. The procedure is defined in photoneo_common_X.script. Do not edit it.

Request	Script definition	Input	Populates
Connect to the Action Request Server	pho_wait_for_server(server_ip, port=PHO_BINPICKING_PORT_DEFAULT, max_attempt=1)	server_ip, port (default 11003), max_attempt (default 1)	PHO_ERR_CODE, system_status

pho_wait_for_server closes the socket first and then opens it again, so a restarted program reconnects cleanly. After the socket opens, it sends the brand ID UNIVERSAL_ROBOTS/v1.12.0. In the example programs, the vision controller address is in the variable IP_ADDRESS (default "192.168.1.1"), set in Before Start.

Note

If the socket cannot be opened, the library sets PHO_ERR_CODE to 3 (communication failure), sets system_status to 501 (disconnected) and stops the program. When the connection is established, the Deployment page of Bin Picking Studio shows the Action Request Client (Robot) as CONNECTED.

3.2 Request List

This section describes the API calls of the Robot module. Use them for high-level control of the bin picking application. For details about each request, see Action requests.

Note

These procedures are defined in the API part of photoneo_common_X.script. Do not edit them. The ID column shows the request ID of the Photoneo protocol. pho_bin_picking is a local procedure. It executes the received trajectory and sends no request.

Request	ID	Script definition	Input	Populates
Initialize	4	pho_request_init(pho_start_pos, pho_end_pos, vision_sys_id=PHO_VISION_ID_DEFAULT, timeout=-1)	Start pose, End pose, vision system ID, timeout in s (optional)	PHO_ERR_CODE, pho_start_pose, pho_end_pose
Scan	1	pho_request_scan(vision_sys_id=PHO_VISION_ID_DEFAULT)	Vision system ID	Non-blocking. Collect the response with pho_wait_for_scan_completion().
Wait for scan completion	-	pho_wait_for_scan_completion()	None	PHO_ERR_CODE
Trigger scan (MultiView)	28	pho_request_trigger_scan(vision_sys_id, tcp_pose=PHO_NO_POSE)	Vision system ID, flange pose (optional): get_actual_tool_flange_pose()	PHO_ERR_CODE
Reuse scan (Localize on the last scan)	29	pho_request_reuse_scan(vision_sys_id)	Vision system ID	Non-blocking. Collect the response with pho_wait_for_scan_completion().
Trajectory	2	pho_request_trajectory(vision_sys_id=PHO_VISION_ID_DEFAULT)	Vision system ID	Non-blocking. Collect the response with pho_receive_trajectory().
Receive trajectory	-	pho_receive_trajectory(timeout=-1)	None. The timeout parameter is accepted but not used.	PHO_ERR_CODE, trajectory buffer and operation list, pho_tool_point_invariance, pho_gripping_point_id, pho_gripping_point_invariance, pho_dimension_x, pho_dimension_y, pho_dimension_rot, pho_nn_label, pho_pick_allowed
Execute pick	local	pho_bin_picking(pho_start_position=pho_start_pose, vision_sys_id=PHO_VISION_ID_DEFAULT)	See section 3.3	-
Pick failed	7	pho_request_send_pick_failed(vision_sys_id=PHO_VISION_ID_DEFAULT)	Vision system ID	PHO_ERR_CODE
Object pose	8	pho_request_object_pose(vision_sys_id)	Vision system ID	PHO_ERR_CODE, pho_object_pose, pho_dimension_x, pho_dimension_y, pho_dimension_rot
Get vision system status	21	pho_request_get_vs_status(vision_sys_id)	Vision system ID	PHO_ERR_CODE, pho_num_localized, pho_num_ready, pho_process_state
Change environment (Change scene state)	15	pho_request_change_env_state(state_id)	Scene state ID	PHO_ERR_CODE
Change bounding box	32	pho_request_change_bbox(vision_sys_id, bbox_id)	Vision system ID, bounding box ID	PHO_ERR_CODE
Calibration start	25	pho_calib_start(solution_id, vision_sys_id)	Solution ID, vision system ID	PHO_ERR_CODE
Calibration add point	5	pho_calib_add_point()	None. The vision controller reads the flange pose over RTDE.	PHO_ERR_CODE
Calibration save	27	pho_calib_save()	None	PHO_ERR_CODE, pho_calib_acc, pho_camera_pose
Calibration stop	26	pho_calib_stop()	None	PHO_ERR_CODE
Change solution	9	pho_request_change_solution(solution_id)	Solution ID	PHO_ERR_CODE
Start solution	10	pho_request_start_solution(solution_id)	Solution ID	PHO_ERR_CODE
Stop solution	11	pho_request_stop_solution()	None	PHO_ERR_CODE
Get running solution	12	pho_request_running_solution()	None	PHO_ERR_CODE, pho_running_solution
Get available solutions (deprecated)	13	pho_request_available_solutions()	None	PHO_ERR_CODE, pho_available_solutions
Bin locator (not supported yet)	3	pho_request_bin_locator(vision_sys_id=PHO_VISION_ID_DEFAULT)	Vision system ID	PHO_ERR_CODE
Communication check	100	pho_request_comm_check()	None	PHO_ERR_CODE

Three requests are non-blocking, so the robot can continue to move while the vision controller scans, localizes and plans:

• pho_request_scan and pho_request_reuse_scan return as soon as the request is sent. pho_wait_for_scan_completion() collects the response. It blocks until the vision controller confirms the scan.

• pho_request_trajectory returns as soon as the request is sent. pho_receive_trajectory() collects the response.

Every request first waits for a pending scan response, so a request that follows a scan also waits for the scan confirmation.

Note

Only one trajectory request can be pending at a time. If you send a new request before you collect the response of a trajectory request, the library shows a PROGRAM FLOW ERROR pop-up and halts the program. The same happens if you call pho_receive_trajectory() without a preceding pho_request_trajectory().

Note

pho_request_init has an optional timeout in seconds. The default, -1, waits without limit. The library counts the timeout in robot control cycles between socket reads, so use it as a safety net against a lost response, not as an exact deadline.

Changes in version 1.12.1 of the Robot module:

• Planned trajectories are streamed by default. The robot follows a smooth curve through the waypoints, with a speed profile limited by pho_stream_vel and pho_stream_acc, instead of one servoj per waypoint. The robot no longer overshoots waypoints, and its speed no longer depends on the waypoint spacing (section 3.4).

• FINE segments end exactly on the last waypoint. The robot holds it until it is there and at rest, and only then is the gripper procedure called.

• The old execution is kept as PHO_EXEC_SERVOJ. Its gain is lowered from 2000 to 1000, and its slow, medium and fast sets have longer lookahead times.

• The communication check no longer stops the program with a list index error. A scan or trajectory response that times out no longer leaves the program waiting forever.

Changes in version 1.12 of the Robot module:

• pho_request_reuse_scan replaces pho_run_loc_on_last_scan. The old name is kept as a deprecated wrapper. The request is now non-blocking, so call pho_wait_for_scan_completion() after it, the same as after pho_request_scan.

• Info data are identified by a unique key, so a value always arrives with the same key, independent of the request (section 3.5 and Info data). pho_nn_label, pho_max_z_height and pho_tilt were added.

• New response codes: 7 (brand ID error), 8 (Bin Picking Studio in simulation mode), 219 and 222 (section 3.6).

• Gripper commands 6, 7 and 8 (User 4, User 5, User 6) were added to the protocol (section 3.3).

• The receive timeout is counted in control cycle time, not in read attempts, so the value is now in seconds. Joint and pose values sent to the vision controller are rounded, not truncated.

For all changes in the communication interface, see Migration from BPS 1.11.x to BPS 1.12.x.

3.3 Bin Picking Sequence

A complete bin picking cycle has five steps:

1. pho_request_init - once for each vision system, before the first scan. The Start and End poses of this request are the first and the last waypoint of every trajectory that Bin Picking Studio plans for this vision system.

2. pho_request_scan - starts the scan, the localization and the path planning.

3. pho_wait_for_scan_completion - waits until the vision controller confirms the scan request.

4. pho_request_trajectory and pho_receive_trajectory - request and receive the planned trajectory for the next object.

5. pho_bin_picking - executes the trajectory.

The procedures below are defined in the customer definitions script block, except the gripper functions, which are Global Functions of the program. Do not edit pho_bin_picking. Adapt the speed settings (section 3.4) and fill in the gripper functions.

Procedure	Description
pho_bin_picking(pho_start_position=pho_start_pose, vision_sys_id=PHO_VISION_ID_DEFAULT)	Description: Executes the trajectory received by the last pho_receive_trajectory(). It goes through the operation list of the trajectory, executes every trajectory segment and calls the gripper function that Bin Picking Studio requested at each waypoint. Do not edit this procedure. To change the motion settings, see section 3.4. Input parameters: pho_start_position and vision_sys_id are accepted for compatibility with other Photoneo Robot modules, but they are not used. The procedure does not move the robot to the Start pose. Warning: The first waypoint of the trajectory is the Start pose sent with pho_request_init for this vision system. Make sure that the robot is in this pose before you call the procedure. With streamed execution, a robot that is not there is first moved to it with movej. With the old servoj execution, it would try to jump there. Note: Set PATH PLANNING TYPE to PLAN JOINT AND LINEAR TRAJECTORIES on the Settings page of Bin Picking Studio.
pho_bin_picking_simple(pho_start_position=pho_start_pose)	Description: A minimal version for solutions with PATH PLANNING TYPE = PATH PLANNING OFF. It moves to pho_start_pose with movej and then executes exactly four trajectory segments. It calls gripper_detach() before the grasp segment and gripper_attach() after it. It does not use the operation list.
gripper_attach()	Description: User-defined function, typically used to attach the object at the Grasp waypoint. Usage: Called automatically when the Attach procedure is set on the waypoint of the grasping method that the robot has just reached.
gripper_detach()	Description: User-defined function, typically used to release the object during placing. Usage: Called automatically when the Detach procedure is set on a waypoint of the grasping method. Usually, the robot program calls it at the place pose instead.
gripper_user_1(), gripper_user_2(), gripper_user_3()	Description: User-defined functions for special operations, for example a blow-off pulse or a second vacuum zone. Usage: Called automatically when the matching User command is set on a waypoint of the grasping method.

Note

A trajectory can be executed only once. pho_receive_trajectory() sets pho_pick_allowed and pho_bin_picking clears it. If you call pho_bin_picking without a newly received trajectory, the library shows a Program flow error pop-up and halts the program.

Note

Report a failed pick. If the pick does not succeed, call pho_request_send_pick_failed so that Bin Picking Studio can update its state and plan for the remaining objects. Then move back to the Start pose and scan again.

The execute_gripper procedure of the customer definitions calls the gripper function for the gripper command that Bin Picking Studio requests at a waypoint:

Gripper command	Meaning	Calls
1	Attach	gripper_attach()
2	Detach	gripper_detach()
3, 4, 5	User 1, User 2, User 3	gripper_user_1(), gripper_user_2(), gripper_user_3()
6, 7, 8	User 4, User 5, User 6 (new in the 1.12 protocol)	Not handled by the delivered customer definitions. If your solution uses them, add the branches to execute_gripper. The constants PHO_G_USER_4 to PHO_G_USER_6 are defined in the library.

3.4 Bin Picking Speed and Blending Settings

Bin Picking Studio supports up to 10 trajectory segments in one bin picking trajectory. The default is 4 segments (Start to Approach, Approach to Grasp, Grasp to Deapproach, Deapproach to End). If needed, you can add more segments on the Grasping method page of the BPS solution.

pho_execute_trajectory (called by pho_bin_picking) selects how each segment is executed from the number of joint waypoints in the segment:

• Streamed execution (default, pho_execution_mode = PHO_EXEC_STREAMING) - for segments with three or more waypoints. This is the normal case with path planning enabled. The robot follows a smooth curve through the planned waypoints, with a speed profile limited by pho_stream_vel and pho_stream_acc. A new servoj target is sent in every control cycle. The curve never goes beyond a waypoint, and the speed does not depend on how far apart the planner placed the waypoints.

• Old servoj execution (pho_execution_mode = PHO_EXEC_SERVOJ) - one servoj for each planned waypoint, as up to version 1.12.0. It is kept for compatibility and is controlled by pho_servo_time, pho_lookahead_time and pho_servo_gain.

• movej - for segments with only two waypoints (path planning off, or a linear sampling that is too coarse for a short segment).

All parameters are arrays with one value for each trajectory segment. With the default 4-segment grasping method, index 0 is Start to Approach, index 1 is Approach to Grasp, index 2 is Grasp to Deapproach and index 3 is Deapproach to End.

For streamed execution, the motion is controlled by pho_stream_vel (maximum joint speed along the path, rad/s) and pho_stream_acc (maximum joint acceleration, rad/s²). For example, to slow down the Approach to Grasp segment, decrease pho_stream_vel[1]. Where a CNT segment flows into the next segment, the robot keeps moving through the junction and slows down only as much as the change of direction requires.

At the end of a FINE segment (for example, the grasp), the robot holds the last waypoint until it is within PHO_FINE_POS_TOL of it and at rest. Only then is the gripper procedure called. If this does not happen within PHO_FINE_TIMEOUT, the program continues and logs the remaining joint error. This applies to streamed and to old servoj execution.

For movej, the motion is controlled by pho_acc (rad/s²), pho_vel (rad/s), pho_time (s) and pho_blend (m). For example, to slow down the Approach to Grasp segment, decrease pho_vel[1]. If pho_time is above 0, pho_acc and pho_vel are ignored.

The customer definitions contain three speed sets (slow, medium and fast) for each execution mode. The medium sets are active by default. To select another set, comment out the medium set and uncomment the set you want. Keep pho_stream_lookahead_time, pho_stream_gain and the PHO_FINE_* tolerances at their delivered values:

#-------------------------------------------------------------------------
#------------------- BIN PICKING SPEED SETTINGS --------------------------
#-------------------------------------------------------------------------

#If the PATH PLANNING TYPE = PLAN JOINT AND LINEAR TRAJECTORIES then choose how planned trajectories are executed
PHO_EXEC_STREAMING = 0   # smooth speed profile, robot target updated every control cycle (recommended)
PHO_EXEC_SERVOJ = 1      # legacy: one servoj per planner waypoint
pho_execution_mode = PHO_EXEC_STREAMING

#Parameters for PHO_EXEC_STREAMING, one value per trajectory (index 0 = approach, 1 = grasp, 2 = deapproach, 3 = end ...)
#Max joint speed [rad/s] and max joint acceleration [rad/s^2] along the planned path. Uncomment one group of parameters
#slow
# pho_stream_vel = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]
# pho_stream_acc = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]

#medium speed
pho_stream_vel = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
pho_stream_acc = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]

#fast speed
# pho_stream_vel = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
# pho_stream_acc = [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]

#servoj filter settings used while streaming (lookahead 0.03 - 0.2 s, gain 100 - 2000)
pho_stream_lookahead_time = 0.05
pho_stream_gain = 1000

#Parameters for PHO_EXEC_SERVOJ (legacy). Uncomment one group of parameters
#slow
# pho_servo_time = [0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.3]
# pho_lookahead_time = [0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2]
# pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#medium speed
pho_servo_time = [0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15]
pho_lookahead_time = [0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15, 0.15]
pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#fast speed
# pho_servo_time = [0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05]
# pho_lookahead_time = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
# pho_servo_gain = [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

#FINE waypoint settling: robot holds the final target until it is within tolerance and at rest
PHO_FINE_POS_TOL = 0.001     # [rad] max joint error
PHO_FINE_VEL_TOL = 0.01      # [rad/s] max joint speed
PHO_FINE_TIMEOUT = 1.0       # [s] continue anyway (with a log message) after this time

#If the PATH PLANNING TYPE = PATH PLANNING OFF then following parameters control the movements
pho_acc = [1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4, 1.4]
pho_vel = [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
pho_time = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] # if time is specified (pho_time > 0) pho_acc and pho_vel parameters are ignored
pho_blend = [0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02]

Warning

Be careful with blending for movej (PATH PLANNING OFF). If a waypoint is inside the blend radius of the previous or the next waypoint, the robot does not move to this waypoint. This is a problem for precise waypoints such as the grasp. Always keep the blend radius smaller than the distance between the waypoints. movej segments that end in a fine waypoint are executed without blending.

Note

For the first run, keep the delivered medium set and use the Speed slider on the teach pendant to slow down the whole program (see section 4.4).

3.5 Result Variables

The requests do not return values. They write their results to global variables of the library. The robot program can read them at any time after the request. A variable keeps its value until the next request changes it. Info data values (see Info data) are stored in the unit in which they were sent, except pho_dimension_rot, which the library converts to degrees.

Variable	Set by	Meaning
PHO_ERR_CODE	every request	Response code of the last request (section 3.6)
PHO_OCCURRED_ERR	every request	True when the last request returned a non-zero response code
system_status	every request	Request that is being processed, for example 1 while scanning, 0 when waiting, 501 when disconnected
pho_object_pose	pho_request_object_pose	Cartesian pose of the localized object as a UR pose (p[x,y,z,rx,ry,rz], m, rad)
pho_start_pose, pho_end_pose	pho_request_init	Joint angles (rad) calculated from the Start and End poses and sent to the vision controller
pho_pick_allowed	pho_receive_trajectory	True while a received trajectory is not executed yet. pho_bin_picking clears it.
pho_tool_point_invariance	pho_receive_trajectory	Info data key 1 - tool point invariance index
pho_gripping_point_id	pho_receive_trajectory	Info data key 2 - gripping point ID
pho_gripping_point_invariance	pho_receive_trajectory	Info data key 3 - gripping point invariance index
pho_dimension_x, pho_dimension_y	pho_receive_trajectory, pho_request_object_pose	Info data keys 4, 5 - size of the object (mm). Only for solutions that measure the object.
pho_dimension_rot	pho_receive_trajectory, pho_request_object_pose	Info data key 6 - Z rotation of the object, converted to degrees by the library
pho_nn_label	pho_receive_trajectory	Info data key 7 - class label from AI localization
pho_max_z_height, pho_tilt	any response that contains keys 8, 9	Info data keys 8, 9 - stored as sent (mm, mrad). BPS does not send these keys. They are used by Locator Studio.
pho_num_localized	pho_request_get_vs_status	Info data key 12 - number of localized objects
pho_num_ready	pho_request_get_vs_status	Info data key 13 - number of objects ready for picking
pho_process_state	pho_request_get_vs_status	Info data key 14 - 0 when the processing is finished, 1 while the localization is running
pho_calib_acc	pho_calib_save	Info data key 10 - calibration accuracy (µm)
pho_camera_pose	pho_calib_save	Camera pose from the calibration result
pho_running_solution	pho_request_running_solution	Info data key 11 - ID of the solution that is running on the vision controller
pho_available_solutions	pho_request_available_solutions	List of available solution IDs (deprecated request)

3.6 Response Codes

Every request writes its result to PHO_ERR_CODE and sets PHO_OCCURRED_ERR when the code is not 0. The library defines named constants, so the program can compare with the name instead of the number, for example if PHO_ERR_CODE == PHO_NO_PART_FOUND:. Codes 3, 4, 6 and 255 stop the program with a pop-up. All other codes are only reported, and the program continues so that it can react. For the meaning of each code, see Error codes.

Code	Constant	Meaning
0	PHO_NO_ERROR	Request completed successfully
1	PHO_SERVICE_ERR	Service error on the vision controller
2	PHO_UNKNOWN_REQ	The vision controller does not know the request, or the robot received an unknown message
3	PHO_COM_FAILURE	Communication failure. The program stops.
4	PHO_BAD_DATA	Wrong data received. The program stops.
5	PHO_TIMEOUT	The response did not arrive within the timeout
6	PHO_LONG_TRAJECTORY	The trajectory is longer than the trajectory buffer (300 waypoints). The program stops.
7	PHO_BRAND_ID_ERR	The brand ID was rejected (new in 1.12)
8	PHO_BPS_IN_SIMULATION_MODE	Bin Picking Studio runs in simulation mode (new in 1.12)
200	PHO_UNKNOWN_ERR	Unknown bin picking error
201	PHO_PLANNING_FAILED	Path planning failed
202	PHO_NO_PART_FOUND	No object found
203	PHO_NOT_INITIALIZED	The vision system is not initialized. Call pho_request_init first.
204	PHO_PART_LOST	The object was lost between localization and picking
218	PHO_EMPTY_SCENE	Empty scene
219	PHO_UNKNOWN_STATE	Unable to read the vision system pose (hand-eye vision systems; new in 1.12)
220	PHO_SCENE_NOT_AV	Scene state not found
221	PHO_SCENE_ACTIVE	Scene state already active
222	PHO_BBOX_ID_INVALID	Invalid bounding box ID (new in 1.12)
255	PHO_WRONG_BG_CONF	Wrong bin picking configuration. Details are in the Deployment console. The program stops.
450 - 459	PHO_INVALID_SOLUTION_ID … PHO_SOLUTION_FORBID	Solution errors: invalid solution ID, another solution running, start / stop error, solution incomplete, invalid brand ID, solution already running, operation forbidden
460 - 469	PHO_INVALID_VS … PHO_CALIB_NOT_ALLOWED	Calibration errors: invalid vision system, calibration already running / not running, start / stop error, calibration disabled, not enough points, error too high, scanner not available, calibration not allowed
501	PHO_DISCONNECTED	Value of system_status when the connection to the vision controller is down

3.7 Example Programs

The Robot module contains several .urpx example programs. They show how to use the requests from section 3.2 in different use cases. In all examples, the vision controller address is in IP_ADDRESS and the vision system in VISION_SYS_ID.

Example program	Description
pho_main_basic	The basic bin picking template. Start with this program. It shows the complete workflow: initialize the vision system, send a scan request, request and receive the trajectory, and execute it. The next scan is started at the Before_Place pose, before the place motion, so that the localization and planning of the next object run while the robot places the current one. After each request, the program checks PHO_ERR_CODE. For “no object found” and “path planning failed”, it moves to Scanning and scans again. For other errors, it shows a message and halts. Note: Teach the START_VS_1 and END_VS_1 waypoints in the pos function of the Positions module (section 4.1).
pho_main_basic_hand_eye	The same as pho_main_basic, for a sensor on the robot flange. The scan requests are sent from the taught Scanning pose. The vision controller reads the flange pose over RTDE, and the library sets the TCP to the flange during the request, so no pose input is necessary.
pho_main_basic_multi_vs	The same as pho_main_basic, but it switches between two vision systems. VISION_SYS_ID selects the vision system that is initialized, scanned and asked for a trajectory. The switch is at the end of the placing part of the loop. Note: Initialize each vision system with its own Start and End poses (START_VS_1 / END_VS_1, START_VS_2 / END_VS_2).
pho_main_change_enviroment	The same as pho_main_basic, but it switches between two environment (scene) states with pho_request_change_env_state(). ENVIROMENT_ID selects the state for the next switch (1 or 2). The switch is at the end of the placing part of the loop.
pho_main_change_solution	The same as pho_main_basic, with solution requests. At the start, the program reads the running solution with pho_request_running_solution(). If it is not SOLUTION_ID, it stops it and starts SOLUTION_ID with pho_request_start_solution(). After each pick, it toggles SOLUTION_ID between 1 and 2 and calls pho_request_change_solution().
pho_main_change_bb	The same as pho_main_basic, but it switches between two bounding boxes of one vision system. After each pick, BBOX_ID toggles between 1 and 2, and pho_request_change_bbox(VISION_SYS_ID, BBOX_ID) activates the next one. You can switch between preconfigured scanning volumes without changing the solution in Bin Picking Studio.
pho_main_get_object_pose	Similar to pho_main_basic, but the robot asks for the Cartesian pose of the object with pho_request_object_pose() instead of a trajectory, and plans all motions itself. This is useful for pick verification, slip sheet detection and simple picking tasks. The result is stored in pho_object_pose. The program copies the pose to Pick, calculates an approach pose 50 mm above it along the object Z axis (Pick_offset = p[0,0,-0.05,0,0,0]), moves there with movej and then with movel to the object. Note: This example needs a real tool. The pose is approached with the active TCP, so set the TCP to the real gripper and make sure that the Z axis of the localized object points in the approach direction. The program asks the operator to confirm both before the first pick. Note: The pose is the raw localization result. Its origin is the origin of the CAD / STL model. No gripping points and no invariances are applied. Note: There is no collision-free path planning in this mode. The approach and retract motions are simple move commands calculated from the object pose.
pho_main_get_status	The same as pho_main_basic, with pho_request_get_vs_status() calls in the loop (three calls, 1 s apart). You can call the request repeatedly during localization. It returns pho_num_localized (localized objects), pho_num_ready (objects ready for picking) and pho_process_state (state of the vision system). Use these values to decide when to start the pick.
pho_main_reuse_scan	The same as pho_main_basic_multi_vs, with the reuse-scan request. Both vision systems are initialized at the start: VS1 with START_VS_1 / END_VS_1 and VS2 with START_VS_2 / END_VS_2. The main loop alternates between the functions Switch_VS_1 and Switch_VS_2. Switch_VS_1 scans for VS1 and picks. Switch_VS_2 reuses this scan for VS2 instead of scanning again. Use this request when the scene has not changed since the last scan, but the localization must run again with a different configuration (another object, bounding box or settings). Note: The delivered example still calls the deprecated pho_run_loc_on_last_scan(). Use pho_request_reuse_scan() in new programs. The following pho_request_trajectory() waits for the scan response automatically. An explicit pho_wait_for_scan_completion() makes this wait visible.
pho_main_multiview_static	Static MultiView (meshing) example. The program merges several scans before the localization starts. This is useful for complex scenes or large objects. The robot stops at each scanning pose to capture a scan. Procedure: The robot stops at the taught poses Start, Middle and End and calls pho_request_trigger_scan() at each pose. After the last capture, pho_request_scan() starts the localization. This request does not capture a new scan. It only starts the localization on the captured data. Requirements: RTDE must be available during the whole process. The example passes get_actual_tcp_pose() to pho_request_trigger_scan. This is the flange pose only when Tool_flange is active. If you use another TCP, pass get_actual_tool_flange_pose(). Scan limit: Use fewer than 10 scans for the best performance.
pho_main_multiview_dynamic	Dynamic MultiView example (Photoneo Instant Meshing). It uses the Parallel Structured Light technology of MotionCam-3D to capture scans while the robot moves, and merges the data into one point cloud. Implementation: A thread calls pho_request_trigger_scan(VISION_SYS_ID, get_actual_tool_flange_pose()) every CAPTURE_GAP seconds while the MESH_DYNAMIC flag is set. The main program sets the flag before the scanning motion (Start → Middle → End), clears it at the end and then starts the localization with one pho_request_scan(). Requirements: A MotionCam-3D is required. PhoXi 3D Scanners do not support dynamic scanning. The example uses CAPTURE_GAP = 0.1 s. If the system cannot keep up, increase it up to 0.5 s. Trajectory: Keep the scanning motion smooth, avoid sudden rotations and keep the scanned area in the field of view. Scan limit: Use no more than 60 scans.
pho_calibration_basic	Basic calibration example for a static sensor. Requirements: RTDE must be enabled (section 2.2), because pho_calib_add_point() sends no pose. The vision controller reads the flange pose over RTDE. Start and confirm the first calibration of the vision system manually in Bin Picking Studio. Calibration steps: Teach the poses Calib_home and Calib_pose_1 to Calib_pose_9. You can add the points manually with Add Calibration Point in Bin Picking Studio, or call pho_calib_add_point() from the program. Calibration accuracy: The calibration error should be below 3 mm. A higher error usually means a systematic problem in the calibration setup.
pho_calibration_auto	Automatic recalibration example. When the first calibration is successful and automatic recalibration is enabled in the vision system settings, the program controls the full cycle: pho_request_stop_solution(), pho_calib_start(SOLUTION_ID, VISION_SYS_ID), nine pho_calib_add_point() calls, pho_calib_save() and pho_calib_stop(). Set SOLUTION_ID (109 in the example) and VISION_SYS_ID to your values. Important: The calibration object must stay where it was during the first calibration. The calibration ball stays mounted in the same place on the flange or gripper and moves with the robot. The marker pattern stays in the same place in the cell. Otherwise, the automatic recalibration fails.
pho_calib_basic_hand_eye	The same calibration procedure for a robot-mounted (hand-eye) sensor (marker pattern calibration). pho_calib_add_point() sends no pose, so no pose input is necessary and a configured TCP offset is ignored. Note: In the delivered file, the folders 3.Calibration Pose to 9.Calibration Pose all move to Calib_pose_1. Before you use the program, select a different taught pose in each of these Move nodes.

4. Runtime

When the solution is fully configured on the vision controller, finish the remaining steps on the robot side and run the bin picking program.

4.1 Teach Positions

After you open pho_main_basic.urpx or another example, touch up four local poses before you run the program: Scanning, Before_Place, Place and Deaprr_Place. Adapt them to your cell.

You must also define the Start and End poses for all vision systems of the solution. They are the first and the last waypoint of each trajectory, and the Initialize request needs them for each vision system. Usually, you teach them with the robot tool above the center of the bin.

To set them, open the pos function of the Positions module on the right side of the program, select the waypoint START_VS_1 and teach it. Then do the same for END_VS_1.

Note

Do not rename the waypoints. The program calls pho_request_init with START_VS_1.p and END_VS_1.p, so the names connect the program to the vision system.

Note

Teach these waypoints with the Tool_flange end effector active, and make sure that it is active when pho_request_init is called. The request converts the taught Cartesian pose to joint angles with get_inverse_kin(), which uses the active TCP. With another end effector active, the robot is initialized with a different Start pose than the one you taught.

4.2 Gripper Commands

The gripper functions are in the Global Functions module of the program. Fill them in with the IO commands, URCap commands or other actions of your gripper. In the example programs, gripper_attach sets digital output DO 0 to high and gripper_detach sets it to low. The user functions are empty.

On the Grasping Methods page of Bin Picking Studio, you can set a gripper command for each main waypoint of the trajectory. For example, if the Attach procedure is set on the Grasp waypoint, pho_bin_picking calls gripper_attach() when the robot reaches the Grasp waypoint.

4.3 Runtime Prerequisites

Do a final check before you start the bin picking program on the robot.

Make sure that:

• The bin picking solution is correctly configured on the vision controller.

• The network setup on the robot side is complete.

• RTDE is enabled and the Robot State Server shows CONNECTED on the Deployment page.

• All vision systems in the solution are calibrated.

• The Start and End poses of all vision systems are touched up.

• All local poses in the main program are touched up.

• The gripper functions are ready and working.

4.4 Running the pho_main_basic Program

Deploy your solution. Until the robot program connects, the Deployment page shows the Action Request Client (Robot) status as DISCONNECTED.

Note

Before you run the program for the first time, we strongly recommend that you decrease the Speed slider on the teach pendant to 20%.

When the connection is established, the Action Request Client and Robot State Server status changes to CONNECTED. The sensor then captures the first scan, and the localization starts.

If there is a pickable object in the scene and the robot controller has received the trajectory for it, the robot starts to move towards the object.

If everything looks fine, let the robot continue towards the first object and check that the path is correct. If the robot stops too far from the object or pushes it too deep, change the settings on the Tool Point or Gripping Point page of Bin Picking Studio.

If the trajectories look fine, set up your own placing routine and slowly increase the speed back to 100%.

You have successfully deployed the Photoneo Robot module for Universal Robots with PolyScope X. Use the example programs (section 3.7) and the request list (section 3.2) to improve your application.

5. Robot Module Update

To update the Robot module to a version that matches your Bin Picking Studio version:

1. Back up your customer definitions. Open your program, select the second Script node in Before Start, press Edit Script and copy its content. It contains your speed settings and any changes to the bin picking procedures. Also note your gripper functions.

2. Import the new example program. Import the .urpx program of the new module version (section 2.3). It contains the new photoneo_common_X.script and a new copy of the customer definitions. There is no script file to replace on the robot controller.

3. Apply your changes again. Copy your speed settings and gripper commands from the backup into the new program.

4. Check the API changes. Read the API changes of the new Robot module version and update the request calls in your program. The changes in 1.12.1 and 1.12 are listed at the end of section 3.2.

The version of each script block is in its header. The library header says Description: Zebra Technologies script for flexible_communication v.1.12 library and Robot module version: 1.12.1. The customer definitions block has its own version line (Customer Definitions v.1.12.1 in the delivered programs). The version that the Robot module reports to the vision controller is the brand ID UNIVERSAL_ROBOTS/v1.12.0.

Note

Robot module 1.12.1 still sends the brand ID UNIVERSAL_ROBOTS/v1.12.0. The vision controller checks this string when the robot connects and rejects values that it does not know. So the patch release keeps the 1.12.0 brand ID and works with every Bin Picking Studio version that supports 1.12.0.

6. Version and Brand ID

Item	Value
Robot module version	1.12.1 (photoneo_common_X.script v.1.12)
Bin Picking Studio	1.12
Brand ID (PHO_ROBOT_ID_STR)	UNIVERSAL_ROBOTS/v1.12.0
Action Request Server port	11003
Robot state (RTDE) port	30004
PolyScope X	10.12.1 or higher